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ABSTRACT 

While asynchronous discussion forums have been widely implemented across 
different learning environments, there have been mixed findings in terms of their 
impact on student performance. This study aims to investigate the impact of an 
asynchronous lecturer-supported discussion forum used to augment traditional 
Face-to-Face course delivery, on student performance in a Quantitative Methods 
course taught to business students in the Caribbean. An asynchronous support 
discussion forum was set up for a group project designed for students to apply 
quantitative techniques to solve real-world problems. The quantity and quality of 
student participation and lecturer participation in the discussion forum, 
represented by student questions and lecturer questions, were investigated to 
determine their impact on student performance, measured by group project 
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marks for 109 student groups over three academic years. The results showed 
that neither student participation nor lecturer participation in the asynchronous 
support discussion forum led to significantly better student performance by way 
of group project mark. Possible reasons for this finding that is inconsistent with 
the literature include issues such as voluntary student participation in the forum, 
multiple other avenues to obtain lecturer support, and the use of questions to 
represent participation. The study’s conceptual framework points to the 
importance of specifying the learning environment, learning objective, discussion 
forum type and engagement type in examining the impact of a discussion forum. 
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RESUMEN 


Si bien los foros de discusión asincrónicos se han implementado ampliamente 
en diferentes entornos de aprendizaje, ha habido hallazgos mixtos en términos 
de su impacto en el desempeño de los estudiantes. Este estudio tiene como 
objetivo investigar el impacto de un foro de discusión asincrónico, impartido por 
un profesor, que se utiliza para aumentar el rendimiento de los estudiantes en 
cursos presenciales tradicionales en un curso de Métodos Cuantitativos, 
impartido a estudiantes de negocios en el Caribe. Se creó un foro de discusión 
de soporte asíncrono para un proyecto grupal, diseñado para que los estudiantes 
aplicaran técnicas cuantitativas para resolver problemas del mundo real. Se 
investigó la cantidad y calidad de la participación de los estudiantes y la 
participación del profesor en el foro de discusión, representada por las preguntas 
de los estudiantes y las preguntas del profesor, para determinar su impacto en 
el desempeño de los estudiantes, medido por las calificaciones de proyectos 
grupales para 109 grupos de estudiantes durante tres años académicos. Los 
resultados mostraron que ni la participación de los estudiantes ni la participación 
de los profesores en el foro de discusión de apoyo asincrónico condujo a un 
rendimiento significativamente mejor de los estudiantes por medio de la 
calificación del proyecto grupal. Las posibles razones de este hallazgo que es 
inconsistente con la literatura incluyen cuestiones como la participación 
voluntaria de los estudiantes en el foro, muchas otras vías para obtener el apoyo 
de los profesores y el uso de preguntas para representar la participación. El 
marco conceptual del estudio señala la importancia de especificar el entorno de 
aprendizaje, el objetivo de aprendizaje, el tipo de foro de discusión y el tipo de 
participación al examinar el impacto de un foro de discusión. 


PALABRAS CLAVE 


foro de discusión asincrónico apoyado por un profesor, aprendizaje aumentado, 
métodos cuantitativos 


INTRODUCTION 
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Quantitative courses are mandatory in business degrees as business 
graduates are expected to have quantitative reasoning, problem solving and 
analytical skills (Ramos Salazar, 2018). A number of business students, 
however, cope with ‘quantitative anxiety,’ defined as anxiety experienced when 
dealing with any quantitative subject matter (Swart 8 Wuensch, 2016). In addition 
to this anxiety, there is also a negative attitude towards these types of courses in 
the business curriculum, where these courses are often viewed as being the most 
challenging and difficult courses in their degree programmes (Yousef, 2017). 
Even more troubling, is the view that these courses are separate and not relevant 
to their substantive programme of study (Buckley et al, 2015). 

Research shows that business students actively strategise to delay taking 
quantitative courses, switch majors to avoid these courses, and even drop out of 
college or university (Swart & Wuensch, 2016). Despite the unpopularity of these 
quantitative courses among students, the quantitative processing skills that are 
developed by these courses are highly sought after by employers as problem 
solving and numeracy skills are seen as critical for growing and sustaining 
businesses (Lee-Post, 2019). 

As a result, there has been much interest in understanding business student 
performance in quantitative courses, as well as the efficacy of the teaching and 
learning methods used in these courses. In terms of the former, researchers 
such as Yousef (2011) and Yousef (2013) studied the impact of variables such 
as age, gender, nationality, high school type and field of study, on academic 
performance in quantitative courses among undergraduate students. In terms of 
the latter, researchers such as Nilsson and Hauff (2018) identified strategies such 
as integration of quantitative methods within the substantive business subject 
area and practical application of quantitative methods to teach business students. 
Winch and Cahn (2015) considered the use of supplemental tutorial videos in 
teaching management science to business students, and similarly, Laugerman 
and Saunders (2019) examined the relationship between using supplemental 
videos and academic performance in the teaching of statistics to business 
students. Cook, Watson and Vougas (2019) investigated the impact of flexible e- 
learning materials in the teaching of quantitative methods. 

This paper falls into the second category of examining the efficacy of 
asynchronous online discussion forums used in quantitative courses taught to 
business students. Because the asynchronous online discussion forum allows 
for discussions between students and lecturers, and among students, beyond the 
boundaries of the physical classroom (Hew, Cheung, & Ng, 2010), it presents an 
attractive teaching and learning vehicle not only during regular semester 
engagement, but also for periods when Face-to-Face engagement is not 
possible, such as during the COVID-19 pandemic. 

While many other researchers have investigated the efficacy of asynchronous 
online discussion forums, such as the Duncan et al. (2012) study of accounting 
students, much of this work has focused on participative discussion forums in 
online teaching environments. This paper adds to the literature by considering 
asynchronous support discussion forums used to augment Face-to-Face 
delivery, by providing lecturer support to students when they undertake a group 
project, designed for students to apply quantitative methods to a real-life 
scenario. The paper firstly examines students’ participation, by way of student 
questions, in an asynchronous online discussion and their performance and 
secondly, the paper examines the relationship between the lecturers 
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participation, by way of lecturer questions, in an asynchronous lecturer-supported 
online discussion forum and the students’ performance. 

The paper proceeds as follows. Section Two presents the literature review on 
asynchronous online discussion forums and how they contribute to knowledge. 
Section Three outlines the research method used to meet the research aims. 
Section Four presents the research results, while Section Five discusses the 
findings and concludes the paper. 


LITERATURE REVIEW 


Students across various disciplines are required to develop knowledge, skills 
and understanding of quantitative methods. In the short term, while these 
students are in college or university, quantitative techniques prepare students 
with a foundation to tackle many academic tasks. In the longer term, these 
techniques are needed in wider society, in the world of work and in multiple 
research settings (Nilsson & Hauff, 2018). In the world of work, employers seek 
out business graduates who are able to display ‘evidence-based decision-making 
abilities’ (p.68), which require quantitative skills (Hijazi & Zoubeidi, 2017). 
Business students are exposed to tools and techniques that could be applied to 
business decision-making scenarios. These scenarios often require the decision- 
makers to have quantitative numeracy, defined as the skills, knowledge, attitudes 
and problem-solving skills to process and engage in quantitative situations 
(McClure & Sircar, 2008). 

Because there is often a reported mismatch between supply and demand for 
skills in the job market, there has been great interest in how student acquire the 
requisite skills and the integration of these skills for the job market (OECD, 2016). 
For example, Goldfinch (1996) examined the methods used to teach quantitative 
methods to business students, where the effectiveness of ‘school-type classes’ 
was compared to ‘traditional lecture / tutorial method’, and concluded that the 
‘school-type classes’ led to higher examination results. Lin (2018) considered 
whether the use of game play in teaching microeconomics increased 
comprehension, attendance and examination performance among business 
students, and concluded that game play led to increased examination 
performance. Lawrence and Singhania (2004) considered student performance 
when statistics course is taught to undergraduate business students via 
traditional delivery versus distance-learning, where the finding was performance 
was higher when students were in the traditional format versus the distance- 
learning format. Similarly, Verhoeven and Wakeling (2011) considered student 
performance when quantitative methods was taught to business students via 
online delivery versus face-to-face delivery, and concluded that students taught 
using face-to-face delivery enjoyed a higher success rate than students taught 
via online delivery. 

Despite the findings of researchers such as Verhoeven and Wakeling (2011), 
online delivery has increased in popularity over the last few decades. This 
increase in popularity reflects the view that virtual or online learning has 
expanded the bounds of time, place and space that characterise the traditional 
learning environment, based on technology that not only delivers learning 
resources but also enables communication among the learners (Tan, 2017). 

One common tool used in virtual or online learning that enables 
communication among learners is the online discussion forum, where users can 
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obtain feedback on problems, share ideas and interact with other students 
(Romero et al., 2013). These forums can be used via the various delivery modes: 
fully online delivery, blended delivery, and to augment courses that use Face-to- 
Face delivery. Whilst there has been the recognised dichotomy of synchronous 
and asynchronous online discussion forum, the focus of this paper is on the 
asynchronous discussion forum, which Hew et al. (2010) defined as “a text-based 
computer-mediated communication environment that allows individuals to 
interact with one another without the constraint of time and place” (p.572). The 
removal of time and space constraints (Gao et al., 2013) allows for the 
asynchronous online discussion forum to be used in any of the delivery modes 
outlined above: fully online delivery, blended delivery, and to augment courses 
that use Face-to-Face delivery. In particular, the advantages of the 
asynchronous online forum, identified as data retention, accessibility to data, 
flexibility for users with respect to time and opportunities for deeper reflection and 
discussions (Hew, Cheung, & Ng, 2010), present attractive opportunities for 
lecturers to incorporate into their suite of teaching and learning strategies. 
Notwithstanding these advantages, researchers have also reported on 
disadvantages of the asynchronous online forum, such as poor student 
participation and student engagement, feelings of disconnect in the online 
environment, lack of emotional cues, and surface-level discussions (Aloni & 
Harrington, 2018). 

Aside from the categories of synchronous and asynchronous online discussion 
forums, these forums can also be classified based on the objective. As shown in 
Table 1, Gill (2006) outlined five types of online discussion forums: Support, 
Participative Discussion, Task Collaboration, Workflow Management, and 
Administrative. These types of discussion forums reflect the way in which 
lecturers use forums. For example, Coppola et al. (2002) identified three roles 
played by lecturers in asynchronous learning networks: cognitive roles, affective 
roles, and managerial roles. The cognitive role concerns the learning and 
thinking processes, the affective role focuses on relationships among the 
participants and the classroom atmosphere, and the managerial role centres on 
class and course management. 


Table 1. Types of Online Discussion Forums 


Discussion Type Description 

Support Designed to provide answers to general or focused questions 
Participative Discussion Used to host discussion, typically on a focused topic 

Task Collaboration Designed for group collaboration 

Workflow Management Used to track workflows 

Administrative Used for general administrative tasks 


Source: Gill (2006) 


Because simply implementing asynchronous online discussion forums is not 
enough to improve student learning (Parks-Stamm, Zafonte, & Palenque, 2017), 
different approaches have been utilised to examine how these online discussion 
forums contribute to learning. 

One approach taken in examining online discussion forums is the 
categorisation of the message types where, for example, Chen and Chiu (2008) 
proposed a framework for characterising these messages using five dimensions: 
Evaluation, Knowledge Content, Social Cues, Personal Information and 
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Elicitation. Within the Knowledge Content dimension, Chen and Chiu (2008) 
specified three categories: Contribution, Repetition and Null Content. 

In this same vein of studying the nature of the postings, Mazzolini and 
Maddison (2003) classified instructor postings into one of four groups: ‘posing 
questions’; ‘answering questions’; ‘combination of answer and follow-up question’ 
and ‘other types of posts such as administrative housekeeping’. Similarly, 
Blignauta and Trollip (2003) developed a taxonomy of six categories of instructor 
postings, which are classified into either ‘Messages with no academic content’ 
and ‘Messages with academic content’ on asynchronous online discussion forum, 
as shown in Table 2. 


Table 2. Taxonomy of Instructor / Lecturer Postings on Asynchronous Online 
Discussion Forums 


Classification Criteria Description 
Messages with no Administrative | General administrative topics that provide support and 
academic content give directions 
Affective Acknowledgment of learner participation 
Other No academic content and off-task related 
Messages with Corrective Remedial and redirecting messages 
academic content Informative Supportive feedback 
Socratic Reflective questions about learner's posting 


Source: Blignauta and Trollip (2003) 


Another approach to studying learning in online discussion forums is by 
analysing the interaction on the forum. Interaction is viewed as one of the most 
important elements of learning in both face-to-face and online environments 
(Jung et al., 2002). A common typology of the types of interactions on online 
discussion forums comprises learner — learner interaction, learner — instructor 
interaction and learner — content interaction (Lin, Zheng, & Zhang, 2017), and 
these three types of interactions have often been studied to investigate learner 
satisfaction. Similarly, Jung et al. (2002) considered academic, collaborative and 
social interactions on online forums on learner satisfaction and learning. 

In keeping with this theme of interaction, the Interaction Analysis Model 
considers the quality of the interactions and the quality of the learning experience 
in online discussion forums. The model which specifies five phases of knowledge 
construction (Gunawardena et al., 1997; Hew & Cheung, 2011) and is 
summarised in Table 3. 


Table 3. The Interaction Analysis Model 


Phase Description 
Phase 1 Shares and compares information 
Phase 2 Discovers inconsistency of ideas, concepts, or statements 
Phase 3 Negotiates ideas, suggests new construction on issues where conflict exists 
Phase 4 Tests proposed synthesis or co-construction against existing cognitive schema, 
personal experiences, or literature 
Phase 5 Summarises agreement or application of newly constructed meaning or ideas 


Source: Gunawardena et al. (1997) 


Journal of Management and Business Education 6(1), 122-141 128 


Researchers have also examined interaction from the perspective of patterns 
and content. Zhu (2006) used the Analytical Framework for Cognitive 
Engagement in Discussion, summarised in Table 4, to perform content analysis 
of asynchronous discussion forums, where five categories were used for coding: 
Question, Statement, Reflection, Mentoring, and Scaffolding. 


Table 4. Analytical Framework for Cognitive Engagement in Discussion 


Category of Type Characteristics 


Engagement 
Question Type 1 Seeking information through question with a direct and correct 
answer 
Type 2 Inquiring or starting discussion which there is no direct or correct 
answer 


Statement | Type 1 Responding to previous message 
Type 2 Informative through statement that provide information 
Type 3 Explanatory through statement that presents factual information 
Type 4 | Analytical through statement that offers analytical opinion 
Type 5 Synthesizing through statement that summarises messages 
Type 6 Evaluative through statement that presents evaluative judgment 
Reflection | Type 1 Reflective of changes 
Type 2 Reflective of using cognitive strategies 

Mentoring | Type 1 Mentoring which explains concept 

Scaffolding | Type 1 Scaffolding which guides by offering suggestions 
Source: Zhu (2006) 


Of these five categories of engagement, the Question category features in 
research investigating how characteristics of discussion forums impact learning 
outcomes (Liu, Cheng, & Lin, 2013). Noce et al. (2014) pointed out that instructor 
questions are the ‘driving force’ in student engagement and learning on online 
asynchronous discussion forums, and distinguished between authentic 
questions, which are questions that the instructor does not know the answer, 
versus test or display questions, which are questions that the instructor already 
knows the answer. Similarly, Paoletti et al. (2018) classified four types of 
questions: factual, probing, generative and orienting, where factual questions ask 
students about facts, rules and procedures; probing questions ask students to 
explain concepts; generative questions ask students to provide additional 
information that is not factual; and orienting questions direct students to possible 
solution strategies. King (1994) specified three types of questions used by 
students in knowledge construction: integration questions, comprehension 
questions and factual questions. According to King (1994), the Integration 
Question connects ideas or requests explanation; the Comprehension Question 
requests descriptions or definitions; and the Factual Question requests recall of 
facts. Further, as a result of this questioning, integration questions lead to 
knowledge integration, comprehension questions lead to knowledge assimilation, 
and factual questions result in knowledge restating (King, 1994). 

Ultimately, the evaluation of the efficacy of the asynchronous discussion forum 
is not an easy undertaking, and while various measures have been utilised, they 
are viewed as only approximations in terms of understanding if the use of 
technology in this manner benefits the educational process (Gill, 2006). Students 
can provide self-reported learning reports through the use of surveys (Benbunan- 
Fich, 1999). Lecturers can evaluate the actual content of the online discussion, 
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through tools such as content analysis rubrics (Gao et al., 2009) or taxonomies 
such as Bloom's taxonomy (Bloom, 1956) applied to the asynchronous 
discussion transcript (Abawajy, 2012) or through data mining to identify 
relationships and patterns of the posts on the forum (Andresen, 2009). Another 
approach lecturers can take towards evaluating the efficacy of asynchronous 
discussion forum is measuring student performance in a specific course section 
or in the overall course (Duncan, Kenworthy, & McNamara, 2012). 

The type of delivery environment, along with the type of discussion forum 
would dictate the suitability of the approach taken for coding the forum 
interactions and analysing if and how the online discussion contributed to meeting 
the learning objective (Liu, Cheng, & Lin, 2013). Figure 1 shows the conceptual 
research framework used to guide this research undertaking, which was 
developed via synthesis of the existing literature. 


Figure 1. Conceptual Framework for Analysing the Efficacy of Asynchronous 
Online Discussion Forums 


LEARNING ENVIRONMENT | 


Learning Forum Design Forum Implementation Evaluation of Learning | 
Interaction Student Lecturer Subjective Objective 
Type Engagement Engagement 
Student to 
Student 
Content 


Objective 


Student to 
Content Analysis 


Student to Pattern 
Lecturer 


Based on the conceptual framework, the research context for this paper is 
outlined in Figure 2, where the learning environment, learning objective, forum 
design, forum implementation and evaluation of learning are presented. 


Figure 2. Framework Outlining Research Context for Research 
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LEARNING ENVIRONMENT 


Face-toFace, Augmented by Asynchronous Online Discussion Forum 


Learning Forum Design Forum Implementation Evaluation of Learning 
Objective 


Interaction Student Lecturer Objective 
Type Engagement Engagement 
ae d Student Performance: 
Application Support e sols a Student Questions e Group Project 
SEER Lecturer Questions e Final Examination 


METHODOLOGY 


The study was conducted in a Department of Management Studies (DOMS) at 
The University of the West Indies, St. Augustine Campus. The course in the 
study is titled Quantitative Methods and it is a compulsory course for all students 
pursuing any of the nine undergraduate degree programmes offered by the 
DOMS. The course is also read by students from other departments since 
Quantitative Methods is a pre-requisite for a number of courses, including 
Production and Operations Management and Operations Planning and Control. 

The class uses the traditional Face-to-Face delivery mode over 13 weeks, with 
three weekly contact hours. Topics covered over the semester include 
forecasting, decision theory, linear programming, queuing theory, game theory 
and simulation modelling. The course is taught using PowerPoint Presentations, 
videos and worked problems on the smart board or white board. Additionally, 
students are given exercises during the Face-to-Face sessions, and homework 
problems to work through to reinforce learning. The course is assessed via 40% 
coursework and 60% final examination. The coursework comprises two 
individual quizzes, worth 10% each, and one group project, which accounts for 
20% of a student's final grade. All members of the group receive the same mark 
for the group project. 

The group project officially starts in Week 4 of the course and is due in Week 
12. An asynchronous support forum discussion page is set up on Moodle, the 
Learning Management System (LMS) at the university, which is designed to 
provide additional support to the students as they work through the group project. 

The project outline is provided below. 

Quantitative Methods (QM)) focuses on the tools and techniques 
that may be applied in problem solving and decision-making 
scenarios. The objective of this QM Group Project is to engage 
students in the scientific approach to problem solving, where you will 
identify and analyse a real-world problem or decision-making scenario 
and make recommendations to solve the problem or arrive at an 
optimal solution. The real-world scenario could be based on a campus 
issue (eg: student parking on campus, student shuttle service), or an 
off-campus issue (eg: gas-station service waiting time). The project will 
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employ the quantitative analysis approach, using the following four 
steps: 

|. Definition of Problem 

Il. Development of Model 

Ill. Data Collection 

IV. Development of Solution 
Using the tools and techniques covered in QM, you will present an 
analysis of the identified problem. Based on your analysis, you will 
present solutions, which would be practical proposals that are based 
on sound quantitative analysis principles. 


Groups are self-selected and comprise between two — six students. Further, 
each group is encouraged to submit a project proposal, indicating the real-world 
problem or decision-making scenario and the planned approach towards problem 
solving. The Lecturer then provides feedback on each proposal received and 
responds to each post made on the discussion forum. 

This paper reports on three years. In Year 1, there were 36 groups, comprising 
193 students: 61 male students and 132 female students. In Year 2, there were 
31 groups, comprising 166 students: 58 male students and 108 female students. 
In Year 3, there were 42 groups, comprising 207 students: 64 male students and 
143 female students. In total, there were 109 groups. 

In terms of student participation in the asynchronous discussion forum, the 
following hypotheses are tested: 


Hypothesis 1: The quantity of student participation in the form of questions 
in the asynchronous lecturer-supported discussion forum will influence the 
student performance in the group project. 

Hypothesis 2: The quality of student participation in the form of questions in 
the asynchronous lecturer-supported discussion forum will influence the student 
performance in the group project. 


Zhu (2006) identified questions as one type of major interaction on discussion 
forums. Further, as per King (1994), student questions are used in knowledge 
construction, and the different types of questions serve different purposes. As 
such, student questions is used to examine student participation. 

In Hypothesis 1, the independent variable ‘quantity of student participation’ 
(StuQuan) is the aggregate number of student questions made on the group 
discussion forum. In Hypothesis 2, the independent variable “quality of student 
participation” (StuQual) is the rating of student questions, where factual questions 
are classified as level 1, given a score of 1; comprehension questions are 
classified as level 2, given a score of 2; and integration questions are classified 
as level 3, given a score of 3. An example of each type of student questions is 
provided below: 

= Student Factual Question: Is this a multi-phase system? 

= Student Comprehension Question: What is meant by configuration of 
the queuing system? 

= Student Integration Question: Can the average time spent in line be 
used as a reflection of the efficiency of the workers catering to the line? 
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Student questions that had no academic content, such as questions seeking 
administrative answers, such as ‘What time is the project due?’ are classified as 
Level 0 and given a score of 0. 

The dependent variable for both hypotheses is student performance, which is 
measured by the group project mark (GroProj). The group project is scored out 
of a total of 20 marks. Based on the grading scheme used by the university, the 
marks are categorised into the following grade bands: A Grade: 16 — 20 marks; 
B Grade: 13 — 15 marks; C Grade: 10 — 12 marks; F Grade: 0 — 9 marks. 

Noce et al. (2014) highlighted the critical nature of lecturer questions in driving 
engagement on discussion forums. Further, Paoletti et al. (2018) specified the 
different types of lecturer questions in stimulating different student responses and 
outcomes. As such, lecturer questions is used to examine lecturer participation. 

In terms of lecturer participation in the asynchronous discussion forum, the 
following hypotheses are tested: 


Hypothesis 3: The quantity of lecturer questions in the asynchronous 
lecturer-supported discussion forum will influence the student performance in the 
group project. 

Hypothesis 4: The quality of lecturer questions in the asynchronous 
lecturer-supported discussion forum will influence the student performance in the 
group project. 


In Hypothesis 3, the independent variable ‘quantity of lecturer questions’ 
(LecQuan) is the aggregate number of lecturer questions made on the group 
discussion forum. In Hypothesis 4, the independent variable ‘quality of lecturer 
questions’ (LecQual) is the rating of lecturer postings, where factual questions 
are classified as level 1, given a score of 1; probing questions are classified as 
level 2, given a score of 2; generative questions are classified as level 3, givena 
score of 3; and orienting questions were classified as level 4, given a score of 4. 
An example of each type of lecturer question is provided below: 

= Lecturer Factual Question: What are your decision variables, and your 
constraints? 

= Lecturer Probing Question: Why are you using decision analysis in this 
scenario? 

= Lecturer Generative Question: How did you determine that the southern 
carpark is underutilised? 

= Lecturer Orienting Question: Are you confident that you have collected 
sufficient data for your analysis? 


Lecturer questions that had no academic content, such as questions seeking 
administrative answers, such as ‘What time do you want to meet?’ are classified 
as level 0 and given a score of 0. 

Again, the dependent variable for both hypotheses is student performance, 
which is measured by the student final mark in the group project. 

The coding of the questions was done by the lecturer and two research 
assistants collaboratively. In cases where there was disagreement with the 
coding level, discussion was used to come to agreement. 

Data analysis was done using inferential tests using PSPP software (Version 
1.4.1). 
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RESULTS 


Student Participation and Student Performance 

Of the 109 student groups, 50 participated in the lecturer-supported 
asynchronous discussion forum by way of factual, comprehension and integration 
questions. As shown in Table 5, the 50 participating student groups (M=13.87, 
SD=2.33) compared to the 59 student groups which did not participate in the 
forum (M=14.74, SD=2.51) demonstrated statistically significant lower group 
marks, t(107) = -1.85, p = 0.03. 


Table 5. Summary Statistics for Participating Student-Groups versus Non- 
Participating Student-Groups 


Participating Student Groups | Non-Participating Student 
Groups 
Number of Groups 50 59 
Mean Group Mark 13.87 14.74 
Standard Deviation 2.33 2.51 


For the 50 student groups which participated in the forum, Student Question 
Quantity had a range of 1 — 8. Low Student Question Quantity was categorised 
as groups that posed 1 — 3 questions, medium Student Question Quantity was 
categorised as groups that posed 4 — 6 questions, and high Student Question 
Quantity was categorised as groups that posed 7 — 8 questions. For these 50 
student groups, the Student Performance by way of Group Mark had a range of 
8 — 18 marks, which were categorised via the grading scheme of A: 16-20 marks, 
B: 13-15 marks, C: 10-12 marks, and F: 0-9 marks. Table 6a summarises this 
data. 


Table 6a. Participating Student Groups: Student Performance and Student 
Question Quantity 


Student Performance 
Student Question Grade A Grade B Grade C Grade F ROW TOTAL 
Quantity 
Low 11 17 10 1 39 
Medium 1 3 4 8 
High 2 1 3 
COLUMN TOTAL 14 21 14 1 50 


A chi-square test of independence was performed to examine the relationship 
between student question quantity and student performance. The relationship 
was not significant, X (6, N = 50) = 4.99, p = .545. 

For the 50 student groups which participated in the forum, the Student 
Question Quality had a range of 1 — 22. Low Student Question Quality was 
categorised as student question quality aggregate score of 1 — 7, medium Student 
Question Quality was categorised as student question quality aggregate score of 
8 - 14, and high Student Question Quality was categorised as student question 
quality aggregate score of 15 - 22. Table 6b summarises the data. 
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Table 6b. Student Performance and Student Question Quality 


134 


Student Performance 
Student Question Grade A Grade B Grade C Grade F | ROW TOTAL 
Quality 
Low 10 17 9 1 37 
Medium 2 3 4 9 
High 2 1 1 4 
COLUMN TOTAL 14 21 14 1 50 


A chi-square test of independence was performed to examine the relationship 
between student question quality and student performance. The relationship was 
not significant, X? (6, N = 50) = 2.80, p = .834. 


Lecturer Participation and Student Performance 

Of the 109 student groups, 54 student groups were engaged by lecturer 
questions in the lecturer-supported asynchronous discussion forum by way of 
factual, probing, generative and orienting questions. As shown in Table 7, the 54 
student groups which had lecturer questions posed to them in the lecturer- 
supported asynchronous discussion forum (M=14.13, SD=2.47) compared to the 
55 student groups who did not had lecturer questions posed to them in the forum 
(M=14.55, SD=2.45) did not demonstrated statistically significant difference in 
group mark, t(107) = -.90, p = 0.19. 


Table 7. Summary Statistics for Lecturer Engagement with Student-Groups 
versus Lecturer Non-Engagement with Student-Groups 


Lecturer Engagement with Lecturer Non-Engagement with 
Student Groups Student Groups 

Number of Groups 54 55 

Mean Group Mark 

Standard Deviation 


Lecturer Question Quantity had a range of 1 — 9. Low Lecturer Question 
Quantity was categorised as lecturer posing 1 — 3 questions, medium Lecturer 
Question Quantity was categorised as lecturer posing 4 — 6 questions, and high 
Lecturer Question Quantity was categorised as lecturer posing 7 — 9 questions. 
For these 54 student groups, the Student Performance by way of Group Mark 
had a range of 8 — 18 marks, which were categorised via the grading scheme of 
A: 16-20 marks, B: 13-15 marks, C: 10-12 marks, and F: 0-9 marks. Table 8a 
summarises this information. 


Table 8a. Student Performance and Lecturer Question Quantity 


Student Performance 
Lecturer Question Grade A Grade B Grade C Grade F | ROW TOTAL 
Quantity 
Low 16 19 7 1 43 
Medium 1 3 3 1 8 
High 2 1 3 
COLUMN TOTAL 17 24 11 2 54 
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A chi-square test of independence was performed to examine the relationship 
between lecturer question quantity and student performance. The relationship 
was not significant, X (6, N = 54) = 6.49, p = .371. 

For the 54 student groups which were engaged by lecturer questions in the 
forum, the Lecturer Question Quality had a range of 1 — 25. Low Lecturer 
Question Quality was categorised as lecturer question quality aggregate score of 
1 — 7, medium Lecturer Question Quality was categorised as lecturer question 
quality aggregate score of 8 - 14, and high Lecturer Question Quality was 
categorised as lecturer question quality aggregate score of 15 - 25. Table 8b 
summarises this information. 


Table 8b. Student Performance and Lecturer Question Quality 


Student Performance 
Lecturer Question Grade A Grade B Grade C Grade F ROW TOTAL 
Quantity 
Low 14 14 4 2 34 
Medium 2 6 6 14 
High 1 4 1 6 
COLUMN TOTAL 17 24 11 2 54 


A chi-square test of independence was performed to examine the relationship 
between lecturer question quality and student performance. The relationship was 
not significant, X (6, N = 54) = 9.43, p= .151. 


DISCUSSION 


The first finding relates to the statistically significant difference in student 
performance of the groups that posed questions in the lecturer-supported 
asynchronous forum versus student groups that did not participate in the forum. 
Specifically, student groups participating in the forum had lower scores. This 
seemingly counterintuitive result may actually reflect that, where there is a lack 
of understanding of the Quantitative Methods concepts, student groups may seek 
out lecture support via the online discussion forum, by way of posing factual, 
comprehension and integration type questions. 

Regarding Research Hypothesis 1, we examined the influence of student 
participation in the form of questions in an asynchronous lecturer-supported 
discussion forum on student performance. The results of the chi square test of 
independence indicated that there is no significant relationship between the 
student question quantity and the student performance dependent variable by 
way of group project mark. Similarly, for Hypothesis 2, there is no significant 
relationship between the student question quality and student performance. 
While these findings were not consistent with previous research related to online 
discussion forums, which reported higher performance by students who post on 
online discussion forums (Cheng et al., 2011; Halabi & Larkins, 2016; Yoo & Kim, 
2014), the findings were somewhat in line with research that indicate that higher 
interaction on discussion forums is not linked to significantly better student 
performance (Davies & Graff, 2005). Of more relevance is the lack of consistency 
of the findings of this research work with that of Duncan, Kenworthy and 
McNamara (2012), which indicated that student engagement, both in terms of 


Journal of Management and Business Education 6(1), 122-141 136 


student quantity and student quality in asynchronous discussion forums had 
positive impact on final examination performance and overall course grades. 

With research hypotheses 3 and 4, we examined the influence of lecturer 
participation in the form of questions in an asynchronous lecturer-supported 
discussion forum on student performance. The results of the chi square test of 
independence indicated that there is no significant relationship between lecturer 
question quantity and student performance as well as, lecturer question quality 
and student performance dependent variable by way of group project mark. Past 
literature had not explicitly examined the role of lecturers on student performance 
measured using objective assessment. Instead, past research such as Blignauta 
and Trollip (2003), Mazzolini and Maddison (2003) and Parks-Stamm et al. (2017) 
focused on lecturer postings on student participation and reported satisfaction. 

The findings from hypotheses 1 and 2 were the most inconsistent result from 
the literature. A number of factors may be attributed to this inconsistent result 
which could be pulled out from the conceptual framework presented in Figure 1. 
Firstly, the asynchronous lecturer-supported discussion forum served to augment 
a Face-to-Face learning environment, where students had multiple other avenues 
to engage with the lecturer for support, such as speaking with the lecturer before 
class and after class, and meetings during office hours. As such, the degree of 
forum participation may be lower in this type of environment as opposed to fully 
online environments, which would show up in the student question quantity. 

Secondly, the learning objective in this study was application of knowledge, 
while analysis and evaluation are the learning objectives of many forums 
described in the literature such as in Abawajy (2012) and Gao et al. (2013). This 
distinction in learning objective would therefore impact the nature of engagement 
on the forum, and perhaps explain the level of student question quantity and 
student question quality. 

Thirdly, in this study, the asynchronous discussion forum was designed as a 
lecturer-supported forum, and not as a participation forum, which is often the type 
of forum reported in the literature. As such, unlike in participation online 
discussion forums which are designed to promote discussion for comprehension, 
critique, construction of Knowledge or sharing knowledge (Gao et al., 2009), in 
this case of the lecturer-supported asynchronous discussion forum, students 
would primarily be motivated to engage in the discussion forum if they believed 
that they needed additional support towards meeting the group project objectives. 
Again, the degree of forum participation may be lower in this type of online 
asynchronous discussion forum, which would show up in the student question 
quantity and student question quality. 

Fourthly, as the forum was designed as a lecturer-supported asynchronous 
forum, student participation was measured by way of student questions, and not 
measured by way of coding of all student messages, which is often the manner 
in which student participation is reported in the literature. The literature often 
reports on guided discussion forums, where students respond to lecturer 
questions, and respond to student posts (Aloni & Harrington, 2018). As such, the 
approach used in this research for assessing student engagement would not 
capture all types of student postings on the discussion forum. 

While the use of online discussion forums is increasing in popularity, the 
literature has shown mixed results in terms of the linkage between these 
discussion forums and student performance. This research study did not support 
the positive relationship with an asynchronous support discussion online forum 
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and student performance, both in terms of student participation and in terms of 
lecturer participation, measured by way of student questions and lecturer 
questions, respectively. These findings may raise the question as to whether it 
is worth setting up and administering asynchronous lecturer-supported 
discussion forums? This study does not produce a definitive answer. On the 
one hand, the asynchronous lecturer-supported discussion forum promotes 
student engagement, and it serves as an additional mechanism to assist in 
student learning. The discussion forum could therefore serve as a critical tool for 
students who self-identify as needing additional support and who may have a 
preference to engage with the lecturer via this online forum medium. On the other 
hand, in the augmented environment, students have multiple means of obtaining 
support, and so, the asynchronous lecturer-supported discussion forum may be 
underutilized, both in terms of the quantity and quality of student participation. 

Future studies on engagement with asynchronous discussion forums may 
move us closer to the answer. This point brings us to the main contribution of 
this research, which was the conceptual framework, which proposes a structure 
to guide the assessment of the evaluation of the efficacy of discussion forums on 
student learning. The conceptual framework proposes the consideration of the 
learning environment, learning objective, forum design, forum implementation 
and evaluation of learning, to examine discussion forums. 
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